Skip to content

rrfs_ci: Merge top of ufs-community/develop#277

Merged
christinaholtNOAA merged 4 commits into
NOAA-GSL:rrfs_cifrom
christinaholtNOAA:merge_dev_again
Feb 7, 2022
Merged

rrfs_ci: Merge top of ufs-community/develop#277
christinaholtNOAA merged 4 commits into
NOAA-GSL:rrfs_cifrom
christinaholtNOAA:merge_dev_again

Conversation

@christinaholtNOAA
Copy link
Copy Markdown
Collaborator

DESCRIPTION OF CHANGES:

Merges in top of ufs-community/develop. Needed for machine-independent testing.

TESTS CONDUCTED:

None.

DEPENDENCIES:

PR #115 in NOAA-GSL:ufs-srweather-app.

christinaholtNOAA and others added 4 commits February 1, 2022 12:33
## DESCRIPTION OF CHANGES: 
These changes move the machine-specific configuration information from the configuration, run time, and test scripts (at least a huge portion of it, with some exceptions for run time environment management on WCOSS machines). The machine-specific information now lives in `ush/machine/*.sh` files. 

In addition, a new bash_util, boolify, was added to clean up `ush/setup.sh`. The function can be called for any boolean flag that can be provided with valid values like `("TRUE" "true" "YES" "yes" "FALSE" "false" "NO" "no")`.
…l_workflow (ufs-community#671)

* Update all NOAA-EMC references to ufs-community

* Remove user-specific UPP directory reference.
## DESCRIPTION OF CHANGES: 

A couple of fixes to get the workflow running on Cheyenne.

 - Remove `module purge` from load_modules_run_task.sh. This no longer causes failures on Cheyenne due to intervening PR ufs-community#650, but it should be removed anyway as it can cause future issues
 - Fixing the number of processors used in the mpirun command for the weather model on Cheyenne. I am honestly not sure how this was ever working, but this change fixes nearly all of the runtime failures currently seen on Cheyenne.

## TESTS CONDUCTED: 
### Cheyenne
Ran a set of WE2E tests on Cheyenne, chosen mostly at random to save core hours (I did ensure that a variety of domains were run so that several different MPI layouts were tested). Most tasks succeed, and all failures (aside from one walltime issue) are also tests that fail on Hera with the current develop branch. See issue ufs-community#673 for more details.

**Successful tests:**
 - grid_CONUS_25km_GFDLgrid_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
 - grid_GSD_HRRR_AK_50km_ics_RAP_lbcs_RAP_suite_GSD_SAR
 - grid_RRFS_CONUS_13km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
 - grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
 - grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v16
 - grid_RRFS_CONUS_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
 - grid_RRFS_CONUS_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta
 - grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_HRRR
 - grid_RRFS_CONUS_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
 - grid_RRFS_CONUS_3km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta

**Unsuccessful tests:**
 - All gfdlmp tests (grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp, grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_2017_gfdlmp_regional, grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp)
 - grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16
 - GST_release_public_v1
   - Hit walltime limit

### Hera, Jet, and Orion
Ran the same set of tests on Hera, Jet, and Orion, with similar results. On Hera the GST successfully completed (though was close to reaching the walltime limit). On Jet, a few tests (grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR, grid_RRFS_CONUS_25km_ics_HRRR_lbcs_HRRR_suite_HRRR, grid_RRFS_CONUS_25km_ics_HRRR_lbcs_HRRR_suite_RRFS_v1beta) failed due to missing initial and/or lateral boundary conditions. On Orion, even more tests failed due to missing ICs and LBCs (grid_GSD_HRRR_AK_50km_ics_RAP_lbcs_RAP_suite_GSD_SAR, grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_RAP_suite_HRRR, grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_2017_gfdlmp, grid_RRFS_CONUS_25km_ics_GSMGFS_lbcs_GSMGFS_suite_GFS_v16).

**To summarize, the only test failures were those that were also seen in develop, and mostly due to missing input files on those platforms.**

## DEPENDENCIES:
This will need to be merged prior to ufs-community/ufs-srweather-app#206

## ISSUE: 
ufs-community#663 has technically already been resolved, but this will fully address that specific issue.
@venitahagerty venitahagerty added the ci-hera-intel-WE Kicks off automated workflow test on hera with intel label Feb 7, 2022
@venitahagerty
Copy link
Copy Markdown

Machine: hera
Compiler: intel
Job: WE
Repo location: /scratch2/BMC/zrtrr/rrfs_ci/autoci/pr/842039610/20220207194149/ufs-srweather-app
Build was Successful
Script /scratch2/BMC/zrtrr/rrfs_ci/autoci/pr/842039610/20220207194149/ufs-srweather-app/regional_workflow/tests/WE2E/setup_WE2E_tests.sh does not exist in repo
Cannot run WE2E tests
If test failed, please make changes and add the following label back:
ci-hera-intel-WE

@christinaholtNOAA christinaholtNOAA merged commit 5b3e445 into NOAA-GSL:rrfs_ci Feb 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-hera-intel-WE Kicks off automated workflow test on hera with intel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants